Integrating Parallel Computations and Sequential Symbol Processing in Neural Networks

The paper presents an analysis of computational processes in neural networks, underlying sequential symbol processing. The main problem addressed in this analysis is that computations in neural networks are massively parallel whereas symbol processing is sequential. It is suggested that the two kinds of processes can be reconciled with each other by the idea of causally constructed representations. Such representations are not assumed to encode symbolic structures and, what follows, structuresensitive procedures are not applied to them. If computations are massively parallel, then a content-sensitivity principle is used in defining consecutive computational steps on such representations. This idea of computations is naturally implemented by a pattern association mechanism appearing in neural networks. we will informally specify the possible extent of parallelism and what the term massively parallel means. Then, basing on these considerations, we will present an analysis of how to model an arbitrary mapping of sequences of symbols by means of a massively parallel style of computations. This analysis will lead us to the idea of causal computations, in which consecutive computational steps are defined by associations on causally formed patterns of activity in a neural network. A simple example illustrating such a style of computations will be presented. 1.1. Sequential symbol processing In the paper we formalise the notion of sequential manipulation of symbolic information as a sequencemapping problem (SMP), which is defined as follows. Let’s consider a discrete dynamical system described by the following equations